The Flan Collection: Designing Data and Methods for Effective Instruction Tuning
https://arxiv.org/abs/2301.13688
We find task balancing and enrichment techniques are overlooked but critical to effective instruction tuning, and in particular, training with mixed prompt settings (zero-shot, few-shot, and chain-of-thought) actually yields stronger (2%+) performance in all settings.
Scaling Instruction-Finetuned Language Modelsを受け、開発をブレイクダウン (Abstract)
https://github.com/google-research/FLAN/tree/main/flan/v2
Brief Review — The Flan Collection: Designing Data and Methods for Effective Instruction Tuning